Journal of Educational Evaluation for Health Professions

Research articles

Negative effects on medical students’ scores for clinical performance during the COVID-19 pandemic in Taiwan: a comparative study: Eunice Jia-Shiow Yuan, Shiau-Shian Huang, Chia-An Hsu, Jiing-Feng Lirng, Tzu-Hao Li, Chia-Chang Huang, Ying-Ying Yang, Chung-Pin Li, Chen-Huan Chen; J Educ Eval Health Prof. 2023;20:37. Published online December 26, 2023; DOI: https://doi.org/10.3352/jeehp.2023.20.37

752 View
71 Download

Abstract PDF Supplementary Material: Purpose
Coronavirus disease 2019 (COVID-19) has heavily impacted medical clinical education in Taiwan. Medical curricula have been altered to minimize exposure and limit transmission. This study investigated the effect of COVID-19 on Taiwanese medical students’ clinical performance using online standardized evaluation systems and explored the factors influencing medical education during the pandemic.
Methods
Medical students were scored from 0 to 100 based on their clinical performance from 1/1/2018 to 6/31/2021. The students were placed into pre-COVID-19 (before 2/1/2020) and midst-COVID-19 (on and after 2/1/2020) groups. Each group was further categorized into COVID-19-affected specialties (pulmonary, infectious, and emergency medicine) and other specialties. Generalized estimating equations (GEEs) were used to compare and examine the effects of relevant variables on student performance.
Results
In total, 16,944 clinical scores were obtained for COVID-19-affected specialties and other specialties. For the COVID-19-affected specialties, the midst-COVID-19 score (88.513.52) was significantly lower than the pre-COVID-19 score (90.143.55) (P<0.0001). For the other specialties, the midst-COVID-19 score (88.323.68) was also significantly lower than the pre-COVID-19 score (90.063.58) (P<0.0001). There were 1,322 students (837 males and 485 females). Male students had significantly lower scores than female students (89.333.68 vs. 89.993.66, P=0.0017). GEE analysis revealed that the COVID-19 pandemic (unstandardized beta coefficient=-1.99, standard error [SE]=0.13, P<0.0001), COVID-19-affected specialties (B=0.26, SE=0.11, P=0.0184), female students (B=1.10, SE=0.20, P<0.0001), and female attending physicians (B=-0.19, SE=0.08, P=0.0145) were independently associated with students’ scores.
Conclusion
COVID-19 negatively impacted medical students' clinical performance, regardless of their specialty. Female students outperformed male students, irrespective of the pandemic.

Equal Z standard-setting method to estimate the minimum number of panelists for a medical school’s objective structured clinical examination in Taiwan: a simulation study: Ying-Ying Yang, Pin-Hsiang Huang, Ling-Yu Yang, Chia-Chang Huang, Chih-Wei Liu, Shiau-Shian Huang, Chen-Huan Chen, Fa-Yauh Lee, Shou-Yen Kao, Boaz Shulruf; J Educ Eval Health Prof. 2022;19:27. Published online October 17, 2022; DOI: https://doi.org/10.3352/jeehp.2022.19.27

1,658 View
116 Download

Abstract PDF Supplementary Material: Purpose
Undertaking a standard-setting exercise is a common method for setting pass/fail cut scores for high-stakes examinations. The recently introduced equal Z standard-setting method (EZ method) has been found to be a valid and effective alternative for the commonly used Angoff and Hofstee methods and their variants. The current study aims to estimate the minimum number of panelists required for obtaining acceptable and reliable cut scores using the EZ method.
Methods
The primary data were extracted from 31 panelists who used the EZ method for setting cut scores for a 12-station of medical school’s final objective structured clinical examination (OSCE) in Taiwan. For this study, a new data set composed of 1,000 random samples of different panel sizes, ranging from 5 to 25 panelists, was established and analyzed. Analysis of variance was performed to measure the differences in the cut scores set by the sampled groups, across all sizes within each station.
Results
On average, a panel of 10 experts or more yielded cut scores with confidence more than or equal to 90% and 15 experts yielded cut scores with confidence more than or equal to 95%. No significant differences in cut scores associated with panel size were identified for panels of 5 or more experts.
Conclusion
The EZ method was found to be valid and feasible. Less than an hour was required for 12 panelists to assess 12 OSCE stations. Calculating the cut scores required only basic statistical skills.

Brief report

Effects of a simulation-based blended training model on nurses’ treatment decision-related knowledge about oral cancer in Taiwan: a pilot survey: Chia-Chang Huang, Shiau-Shian Huang, Ying-Ying Yang, Shou-Yen Kao; J Educ Eval Health Prof. 2021;18:10. Published online May 25, 2021; DOI: https://doi.org/10.3352/jeehp.2021.18.10

5,347 View
296 Download
3 Crossref

Abstract PDF Supplementary Material

The present study aimed to evaluate the effects of virtual reality (VR) simulations combined with bedside assignments on nurses’ self-efficacy in providing pre-treatment educational services. Between March 2019 and November 2020, we conducted a study of VR educational materials that were developed to cover information about the treatment of oral cancers. The effects of the VR simulation, the thinking-path tracking map method, and bedside assignments on the nurses’ treatment decision-related knowledge were evaluated in a ward for oral cancer patients at Taipei Veterans General Hospital, Taipei, Taiwan. The blended training model significantly increased nurses’ familiarity (P<0.01) and confidence (P<0.03) regarding their knowledge of treatments and treatment decision-related knowledge. This model also significantly increased their confidence in their skills in bedside pre-treatment education for admitted oral cancer patients (P<0.002). Oral cancer-specific VR materials enhanced the effectiveness of skills training among nurses in the oral cancer ward.

Citations

Citations to this article as recorded by

The use of simulation-based education in cancer care: a scoping review
Amina Silva, Kylie Teggart, Corey Heerschap, Jacqueline Galica, Kevin Woo, Marian Luctkar-Flude
International Journal of Healthcare Simulation.2023;[Epub] CrossRef
Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
Sun Huh
Journal of Educational Evaluation for Health Professions.2022; 19: 2. CrossRef
Assessing the Financial Sustainability of High-Fidelity and Virtual Reality Simulation for Nursing Education
Michael D. Bumbach, Beth A. Culross, Santanu K. Datta
CIN: Computers, Informatics, Nursing.2022; 40(9): 615. CrossRef

Research articles

Effects of virtual reality training on decreasing the rates of needlestick or sharp injury in new-coming medical and nursing interns in Taiwan: Szu-Hsien Wu, Chia-Chang Huang, Shiau-Shian Huang, Ying-Ying Yang, Chih-Wei Liu, Boaz Shulruf, Chen-Huan Chen; J Educ Eval Health Prof. 2020;17:1. Published online January 20, 2020; DOI: https://doi.org/10.3352/jeehp.2020.17.1

11,041 View
413 Download
33 Web of Science
31 Crossref

Abstract PDF Supplementary Material

Purpose
Senior nursing and medical interns’ lack of familiarity and confidence with respect to practicing universal precaution for the prevention of occupational needlestick or sharp injuries may harm themselves. Trainees’ self-reported needlestick or sharp injury rate was known to be especially high during the first 2 months of internship in Taiwan. This prospective cohort study aimed to assess the effect of newly developed virtual reality (VR) game, which uses Gagne’s learning model to improve universal precaution for needlestick or sharp injury prevention and decrease the rates of needle stick or sharp injuries in new-coming medical and nursing interns in Taiwan.
Methods
From 2017 to 2019, the VR system was developed and applied in training of 59 new-coming nursing and 50 medical interns. Occupational needlestick or sharp injury prevention was sought to be achieved through a game of right and wrong choices for safe or unsafe universal precaution behaviors.
Results
In comparison with medical interns, a higher proportion of nursing interns had past experiences of deep occupational needlestick or sharp injury. Before VR training, the familiarity and confidence for needlestick or sharp injury prevention were higher among nursing interns than medical interns. Trainees with past experiences of deep needlestick or sharp injury exhibited better performance on the accuracy rate and time needed to complete 20 decisions than those without past experiences in VR practice. All trainees showed an improved performance after VR training. A high proportion of trainees reported that the VR-based training significantly decreased their anxiety about needlestick or sharp injury prevention.
Conclusion
This self-developed VR game system using Gagne’s flow improved universal precaution for needlestick or sharp injury prevention and reduced the needlestick or sharp injury rates in the first 2 months of nursing and medical internship.

Citations

Citations to this article as recorded by

Impact of an immersive virtual reality simulator education program on nursing students' intravenous injection administration: A mixed methods study
Yi-Ya Chang, Li-Fen Chao, Wen Chang, Chih-Ming Lin, Yi-Hsing Lee, Abigail Latimer, Misook L. Chung
Nurse Education Today.2024; 132: 106002. CrossRef
Positive Aspects and Potential Drawbacks of Implementing Digital Teaching/Learning Scenarios in Health Professions Using Nursing Education as an Example: A Research Report from Germany
Lydia Pfeifer, Sophia Fries, Alexander Stirner, Lisa Nagel, Christian Cohnen, Leona Aschentrup, Marleen Schönbeck, Annette Nauerth, Patrizia Raschper, Tim Herzig, Kamil J. Wrona
Nursing Reports.2024; 14(1): 468. CrossRef
Combatting the occurrence of needle-stick injuries in a medical school: why is it still an issue?
Franca Keicher, Janina Zirkel, Tobias Leutritz, Sarah König
BMC Medical Education.2024;[Epub] CrossRef
Development of Immersive Virtual Reality Hospital Fire Management and Evacuation Training Program for Nursing Students in Hong Kong
Wendy Wing Chi Wong, Gary Long Hei So
SN Computer Science.2024;[Epub] CrossRef
Virtual Reality Head-Mounted Displays in Medical Education
Thomas Tursø-Finnich, Rune Overgaard Jensen, Lasse X. Jensen, Lars Konge, Ebbe Thinggaard
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare.2023; 18(1): 42. CrossRef
Review of literature: Knowledge and practice of standard precautions by nursing student and teaching techniques used in training
Stephanie Bouget Mohammedi, Caroline Landelle
American Journal of Infection Control.2023; 51(5): 574. CrossRef
Mild simulator sickness can alter heart rate variability, mental workload, and learning outcomes in a 360° virtual reality application for medical education: a post hoc analysis of a randomized controlled trial
Li-Jen Hsin, Yi-Ping Chao, Hai-Hua Chuang, Terry B. J. Kuo, Cheryl C. H. Yang, Chung-Guei Huang, Chung-Jan Kang, Wan-Ni Lin, Tuan-Jen Fang, Hsueh-Yu Li, Li-Ang Lee
Virtual Reality.2023; 27(4): 3345. CrossRef
Exploring Key Factors Influencing Nursing Students’ Cognitive Load and Willingness to Serve Older Adults: Cross-sectional Descriptive Correlational Study
Pei-Lun Hsieh, Yu-Rung Wang, Tien-Chi Huang
JMIR Serious Games.2023; 11: e43203. CrossRef
The Development and Pilot Testing of Immersive Virtual Reality Simulation Training for Prelicensure Nursing Students: A Quasi-Experimental Study
Bethany Cieslowski, Tanya Haas, Kyeung Mi Oh, Kathleen Chang, Cheryl A. Oetjen
Clinical Simulation in Nursing.2023; 77: 6. CrossRef
Metaverse applied to musculoskeletal pathology: Orthoverse and Rehabverse
Juan M. Román-Belmonte, E. Carlos Rodríguez-Merchán, Hortensia De la Corte-Rodríguez
Postgraduate Medicine.2023; 135(5): 440. CrossRef
Evaluating the effectiveness of virtual reality for safety-relevant training: a systematic review
Hans Stefan, Michael Mortimer, Ben Horan
Virtual Reality.2023; 27(4): 2839. CrossRef
Virtual reality simulation for nursing education: effectiveness and feasibility
Debra Kiegaldie, Louise Shaw
BMC Nursing.2023;[Epub] CrossRef
Digital Transformation in Nursing Education: A Systematic Review on Computer-Aided Nursing Education Pedagogies, Recent Advancements and Outlook on the Post-COVID-19 Era
Nevena Kostadinova Dicheva, Ikram Ur Rehman, Aamir Anwar, Moustafa M. Nasralla, Laden Husamaldin, Sama Aleshaiker
IEEE Access.2023; 11: 135659. CrossRef
Assessment of preclinical learning using virtual reality based education for nursing students
Sofía Martín Rodríguez, Marcial F Parrilla, Víctor Álvarez-García, Juan Ramón Pérez-Pérez, María Del Mar Fernández-Álvarez, Rubén Martín-Payo
Educational Media International.2023; 60(3-4): 209. CrossRef
How, for Whom, and in Which Contexts or Conditions Augmented and Virtual Reality Training Works in Upskilling Health Care Workers: Realist Synthesis
Norina Gasteiger, Sabine N van der Veer, Paul Wilson, Dawn Dowding
JMIR Serious Games.2022; 10(1): e31644. CrossRef
Application of computer-based testing in the Korean Medical Licensing Examination, the emergence of the metaverse in medical education, journal metrics and statistics, and appreciation to reviewers and volunteers
Sun Huh
Journal of Educational Evaluation for Health Professions.2022; 19: 2. CrossRef
A Systematic Literature Review of Virtual, Augmented, and Mixed Reality Game Applications in Healthcare
Yu Fu, Yan Hu, Veronica Sundstedt
ACM Transactions on Computing for Healthcare.2022; 3(2): 1. CrossRef
Effectiveness of Immersive Virtual Reality in Nursing Education
Jeeyae Choi, C Elise Thompson, Jeungok Choi, Colette B. Waddill, Soyoung Choi
Nurse Educator.2022; 47(3): E57. CrossRef
Systematic Bibliometric Analysis of Research Hotspots and Trends on the Application of Virtual Reality in Nursing
Junqiang Zhao, Yi Lu, Fujun Zhou, Ruping Mao, Fangqin Fei
Frontiers in Public Health.2022;[Epub] CrossRef
Development and Pilot Testing of a Non-stress Test Virtual Reality Simulator
Li-Fen Chao, Tsuey-Yuan Huang, Debra K. Moser, Fen-Fang Chung, Ying-Tung Lau, Xaviera Xiao
CIN: Computers, Informatics, Nursing.2022; 40(6): 357. CrossRef
The Effect of a Child Model on Breath-Sounds Examination Skills and Satisfaction on Nursing Students
Silpthai Thamruangrit, Sermsri Santati, Jumpee Granger, Dongruethai Buadong, Jatuporn Thongsri
Healthcare.2022; 10(7): 1165. CrossRef
Theories informing technology enhanced learning in nursing and midwifery education: A systematic review and typological classification
Siobhan O'Connor, Stephanie Kennedy, Yajing Wang, Amna Ali, Samantha Cooke, Richard G. Booth
Nurse Education Today.2022; 118: 105518. CrossRef
Tendência temporal dos acidentes por exposição percutânea em um hospital público no Brasil, 2007-2019
Renan Sallazar Ferreira Pereira, Cecília Angelita dos Santos, Adriano Marçal Pimenta
Revista Brasileira de Enfermagem.2022;[Epub] CrossRef
Temporal trend of accidents due to percutaneous exposure in a public hospital in Brazil, 2007-2019
Renan Sallazar Ferreira Pereira, Cecília Angelita dos Santos, Adriano Marçal Pimenta
Revista Brasileira de Enfermagem.2022;[Epub] CrossRef
Influence of Technology in Supporting Quality and Safety in Nursing Education
Gerry Altmiller, Loraine Hopkins Pepe
Nursing Clinics of North America.2022; 57(4): 551. CrossRef
Virtual reality simulations in nurse education: A systematic mapping review
Christian Plotzky, Ulrike Lindwedel, Michaela Sorber, Barbara Loessl, Peter König, Christophe Kunze, Christiane Kugler, Michael Meng
Nurse Education Today.2021; 101: 104868. CrossRef
Virtual Reality in Healthcare Skills Training: The Effects of Presence on Acceptance and Increase of Knowledge
Christian Plotzky, Ulrike Lindwedel, Alexander Bejan, Peter König, Christophe Kunze
i-com.2021; 20(1): 73. CrossRef
A systematic review of immersive virtual reality for industrial skills training
Unnikrishnan Radhakrishnan, Konstantinos Koumaditis, Francesco Chinello
Behaviour & Information Technology.2021; 40(12): 1310. CrossRef
No difference in learning outcomes and usability between using controllers and hand tracking during a virtual reality endotracheal intubation training for medical students in Thailand
Chaowanan Khundam, Naparat Sukkriang, Frédéric Noël
Journal of Educational Evaluation for Health Professions.2021; 18: 22. CrossRef
How to train health personnel to protect themselves from SARS-CoV-2 (novel coronavirus) infection when caring for a patient or suspected case
Sun Huh
Journal of Educational Evaluation for Health Professions.2020; 17: 10. CrossRef
Knowledge, attitude and behavior towards COVID-19 among the Turkish healthcare workers
Yasin Uzuntarla, Sumeyra Ceyhan
Journal of Clinical Medicine of Kazakhstan.2020; 6(60): 93. CrossRef

Dreyfus scale-based feedback increased medical students’ satisfaction with the complex cluster part of a interviewing and physical examination course and improved skills readiness in Taiwan: Shiau-Shian Huang, Chia-Chang Huang, Ying-Ying Yang, Shuu-Jiun Wang, Boaz Shulruf, Chen-Huan Chen; J Educ Eval Health Prof. 2019;16:30. Published online October 11, 2019; DOI: https://doi.org/10.3352/jeehp.2019.16.30

10,249 View
126 Download
1 Web of Science

Abstract PDF Supplementary Material: Purpose
In contrast to the core part of the clinical interviewing and physical examination (PE) skills course, corresponding to the basic, head-to-toe, and thoracic systems, learners need structured feedback in the cluster part of the course, which includes the abdominal, neuromuscular, and musculoskeletal systems. This study evaluated the effects of using Dreyfus scale-based feedback, which has elements of continuous professional development, instead of Likert scale-based feedback in the cluster part of training in Taiwan.
Methods
Instructors and final-year medical students in the 2015–2016 classes of National Yang-Ming University, Taiwan comprised the regular cohort, whereas those in the 2017–2018 classes formed the intervention cohort. In the intervention cohort, Dreyfus scale-based feedback, rather than Likert scale-based feedback, was used in the cluster part of the course.
Results
In the cluster part of the course in the regular cohort, pre-trained standardized patients rated the class climate as poor, and students expressed low satisfaction with the instructors and course and low self-assessed readiness. In comparison with the regular cohort, improved end-of-course group objective structured clinical examination scores after the cluster part were noted in the intervention cohort. In other words, the implementation of Dreyfus scale-based feedback in the intervention cohort for the cluster part improved the deficit in this section of the course.
Conclusion
The implementation of Dreyfus scale-based feedback helped instructors to create a good class climate in the cluster part of the clinical interviewing and PE skills course. Simultaneously, this new intervention achieved the goal of promoting medical students’ readiness for interviewing, PE, and self-directed learning.

First
Prev
Page of 1
Next
Last

Search